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ABSTRACT 

For* decades there have been\ atteipts to develop tests 
that ar€ fair to people from different backgrounds. The Fundamental 
Achieveient Series (FAS) is a recent attempt to develop a test* that 
is fair to those considered disadvantaged. In addition to the 
traditional kinds of items, the FAS incorporates relevant items, 
tapping verbal and numerical abilities required in common everyday 
situations^ A comparison of responses to relevant items and 
traditional ones suggests that, when testing disadvantaged adults, it 
is important to use the more relevant items. Item analysis indicated 
that on the SkS numerical test, the relevant items vere easier, yet 
had higher discrimination indices, on the verbal test, the relevant 
items worked as veil as the traditional items. The FAS is not free of 
cultural Mas; differences in test performance do occur between 
blacks and whites. Performance on both parts of the FAS is related to 
reading ability. However, adults with poor reading skills have an 
advantage on the FAS that is not present on traditional paper and 
pencil tests: the instructions for the FAS are played cn a tape 
recorder^ (Author/CTM) 
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ABSTRACT 



The Effectiveness of Relevant Item Content 
in Reducing Test Bias 

For decides there have been attempts io develop tests that are fair to 
people from differing backgrounds. The Fund? mental Achievement Series 
(FAS) is a recent attempt to develop a test that is fair to those considered 
disadvantaged. In addition to the^ traditional hinds of items, the FAS incor- 
porates relevant items^ tapjJng verbal aaid numex^ical abilities required in 
common everyday situations. A comparison of responses to relevant items 
and traditional ones su|^j^;e3ts that, when testing disadvantaged persons, at is 
important lo use the more relevant Idnds of it jms. Item analysis indicated 
that on the FAS Nuinerj.cal tust the more relevant items worked psychometrically 
better than the traditiojial ones, i. e. , were easier yet had higher discrimination 
indices. On the Verbal test, the relevant items wo^^ked as v>'ell as the tradi- 
tional items. It was noted, however, that the FAS is not free of so-called 
cultural bias: differences in test performance do occur between Blacks and 
Whites; als,o, performance on both parts of the FAS is related to reading 
ability. 
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TlVe Effectiveness of Relevant Item Content 
in Reducing Test Bias 

There have been numerous attempts in the past to develop tests that 
would be considered "culture-free. " For a number of reasons such attempts 
have not been successful: Tests are in effect samples of behavfor iiv^cicty; 
thus, past e:rperience docs influence present functioning on tests. To eliminate 
items that are related to experience would leave you with a test that would be 
measuring unimportant, trivial behavior. " . 

A more recent trend has been to develop tests that include items con-- 
sidered fair to all groups being tested, (e^g. , the advantaged and disadvantaged, 
males and females). To a large extent the content of these tests is more rele- 
vant to tiie cveryaay experiences of the persons tested. The content of the items 
depends upon cultural influences, but as much as possible, the items are 

i 

supposed to be equally familiar to jail, groups concerned. This means that 

either the g^roups perform equally ^well on all items, or that an equal number of 

I 

items favoring specific groups appear in the test. These tests do not attempt 
to eliminate cultural influences, but attempt to sample cultural experiences 
that are common to those being tested. 

Such tests, if successful, would be extremely useful in vocational and 
educational guidance. It is important, however, that these new tests be 
examined carefully to see if they indeed function in the way the test publishers 
say they do. The purppse of the present study was to compare the effectiveness 
of so-callcJ relevant items To items traditionally found in paper-aiid-pencil tests. 



Tlie Fundameiital Achievement. Series, 

The specilic lest of interest in this paper is the Fiuidanicntal Achievement 
Series (FAS) published by the Psychological Corporation (1968), aiid revievvcd 
by Lewis R, Aiken in (he Journal of Educati o nal Ivlea53urement (1970). 

The FAS contains a Verbal and Numerical section, each of Avhich can be 
Mininisiexe.dJ.n_3jO_jnin^^^^ is usbd for presenting the 

questions and directions, and for controlling the tiniing^^ie tape has tlie 
advantage of eliminating the necessity for an examinee to read inct ructions. 

The F/74S is dorJ;^ned to cover a range of ability from basic literacy to 
soniev/hat above cightn j;rade. The manual states tliat (he tests are intended ' 
for use in the employiuent or educaiional placement of adults and adolescents 
who may not ha.ve hp.d the usual exposure to formal education. ' , ^ 

\ 

The Verba] test includes items that measure the ability to read signs, ' 
to add prices that appear on a restaurant menu, to find npjiies Oii apartment 
house lists, to find numbers in a telephone book, to copy sentences in long 
liand, and to understand and remember details from brief oral announcements^ 

More than half of the test, however, includes the more traditional kinds of 

/ 

items found in paper-and-pencil tests: recognizing misspelled v;ords, 
selecting the synonyms for given words, and picture vocabulary. 

The Numerical test Licludcs items that measure the ability to read and 
write numborj^, to tell time, to add the amount of coins, to Imov/ simple facts, 
such as the number of inches in a foot, to measure with a ruler, to calculate 
the area of a rectangle, to aud a sales slip, to\\vrite a check, to read a 
calendar, and to read cliarts. In addilion, tliirty of the sixty-nine items in 



the Numerical test are similar Lo those louiid in traditional paper- aiid-pojicil 
tests: ariuimctic computation, and readmg problems iiivolving arithmetic 
reasoning. 

The authors of the test feel that the inclusion of items related to every- 
day activities should increase the motivation of testees. In addition, although 
overall test scores are more reliable thaii performance on individual items, 
for diagnostic informg,tion attention might be given to certain parts of {he test. 
For example, certain jobs might require an applicant to talie messages and to 
write legibly, or to add up a restaurant check. Such diagnostic use of the test, 
however, is limited, since a small number of items represents only a small 
sample of a person^ s behavior, and may not be a reliable indication of a 
person^s ability. 

Using the FAS in a rohabilifation facility: A study . 

There is still some question as to whether or not the FAS has any 
advantage over other traditional tests used with disadvantaged persons. To 
examine this question the FAS was given to a sample of clients (n = 61) in a 
comprehensive rehabilitation center in New Yolrk City. All clients tested 
were, at the center for the purpose of A^ocational evaluation. A little more 
than haK of. the clients (59%) were women; 44% were Black, and 30% were 
Spanish- spealdng. The ages ranged from less than 20 to 56 years, v;ith more 
than half of the clients (57%) between 20 to 34 years of age. Over half of the 
clients (Gl%) had between 10 to 12 years of education. 



It was not clear from the testing to what extent the FAS was able to 
increase motivation and reduce frustration^ To a large extent the FAS was 
still seen %s a paper-and-pencil test, and had some of the limitations of such 
tests* 

Many clients had difficulty marking the responses in the appropriate 
spaces. Although testees are to write in the booklets and not on answer 
sheets, come clients still had difficulty marking correctly. Foi: example, 
some put X^s on the pictures in the picture vocabularly section rather than 
in the box underneath the picture. Tiiese kinds of errors were common, re- 
flecting a lack of experience in test taldng, and perhaps a difficulty in under- 
standing, and following instructions. 

Another problem area is the tape recording itself. In a few instances 
the actual pronunciation is not clear (e. g. , *^Cliff Street'^ on the bus sign). 
Also the person^ s voice has a definite regional accent, which was commented 
upon by the New York City clients. It is not laiown what effect an unfamiliar 
accent might have on attention and comprehension. 

Also, despite the use of the tape recorJcr for giving instructions, both 

/ 

the Verbal and Numerical parts of the FAS rely to a large extent on reading 
ability. The manual reports a correlation of . 94 (n = 43)Jp.etween the FAS 
Verbal and the Gates-MacGinitie Reading Test. Co,rrelations between the 
FAS Verbal test and the Verbal Reasoning test of the* Differential Aptitude 
Tests (DAT) for eighth graders ranged from . 43 - . 57. Correlations 
between the FAS Numerical test and the Verbal Reasoning test of the DAT 
i^anged from . 22 - . 64. The high^correlations with traditional tests may 



in part reflect the large number of traditional items included in the FAS* 

Comparison of relevant aiid traditional items: Verba l 

Tlie question tlien arises: How successful are the more relevant items 
on the FAS, as compared to the more traditional items? Table 1 may serve 
to answer this question. 



Insert Table 1 about here 



Table 1 reports the median level of difficulty for sets of items measuring 
certain sldlls and also the median discrimination index (point-biserial r) for 
each set of tiems. The range of the difficulty and discrifnination indices for 
the sets of items is also given. 

Thus,' on the first six items of the Verbal test 80% of the clients, on 
the average, were able to read signs, such as **Smoldng Prohibited. " ^The 
actual percentages of persons getting tlie items correct ranged from 67% - 
87% over the six items. The average point-biserial r, using the total score 
on the Verbal te^t as criterion, was . 54. These results indicate that for 
these six items, persons having tfie highest total scores on the F/vS Verbal 
test also tended to be the ones to get these six items correct. Thus, these 
particular itgms seem to be doing their job, that is, identifying persons with 
good verbal ability. 
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The overall conclusion for examining Table 1 is that the relevant items 
(items 1-34) on the Verbal test seem to be functioning as well as the more 
traditional items (items 35-100). Items 25-34 seemed to be quite difficult, 
however, ,and do not discriminate as well as the rest of the items in the test. 
Ill this part of the test announcements are made by the voice on the tape, and 
include details, such as the place and time of a social event. After each 
announcement, the examinees are asked to recall specific facts. The 
questions about the announcements require attention to detail, concentration, 
verbal comprehension, and short-term memoryp Performance could easily 
have been affected by anxiety level. 

The traditional items where one is to select the synonym of a word 
were also somewhat difficult, but did v/ork well as far as identifymg the 
more verbally proficient clients. 

In summary, qn the Verbal test the inclusion of more relevant material 
may not contribute any more psychometric information than conventional items; 
on the other hand, the inclusion of relevant items did not adversely affect lest 
performance. It is possible that motivational level may have been influenced 
by the use of the more relevant content, but such content has not been sbov/n 
to have had a differential effect on performance on the Verbal test. 

Comparison of relevant and traditional items: Numerical 

For the Numerical test another picture seems to emerge. Tlie easiest 
items and the items with the highect discrhnination indices were those that 
were more relevant and wbjse content was more fainiliar to the examinees 
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(e.g\, telling time, adding coins, recognizing numbers, and general informa- 
tion such as the number of inches in a foot). Reading a mileage chart seemed 
to be very difficult, although the clients were able to road a postal chart and a 
sales chart. However, the items involving chart reading in general did not 
discriminate well between those with high and lov/ total scores on the 
Numerical test. 

The traditional reading problems (arithmetic reasoning) were very 
difficult, with only 12% getting an item correct o:"^ the average. These problems 
reflect both reading ability, as well as level of arithmetic r^'asoning. Per- 
formance on such tests is very much influenced \y§ educational experience and 
training. 

Althougli more research needs to be done in this area, it appears that 
for disadvantaged adults it is important to include items on tes* ^ of numerical 
ability that are directly related to everyday experience. Such items seem to 
be better for identifying persons who can work in areas requiring some ability 
to work v/ith numbers. Such items may also be useful for selecting persons 
who could bejiofit by additional schooling or remedial instruction. 

Summary: Using (he FAS in vocational and educational guidanc e. 

The FAS has been published as a research edition and rightly so. More 
data must l^e collected on this test before its value in the field can adequately 
be judged. A revised form will undoubtedly be published at a later date. 
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The FAS is a good a^ttcnipt (o construct aii achievement test suitable 
for disadvantaged adults. The F-A^ has tlie advantage that it is easy to 
administer and takes only an hour of the client^ s lime. Thus, it is a relatively 
quick and efficient way of ti^tting an o*stimj.\te of a client^ s basic verbal and 
numerical ability; a combined score cai^ give \m indication of the person's 
general level of achievement. It should be noted that in vocational counseling 
a verbal and numerical score are not aeces.sarily sufficient to predict success 
in one job as compared to anotlior.^ Such scoics do, however, give an indica- 
" tion of a por^:un^^; present level of ability and su5;geGt possible levels of 

entry for jobs or further opportunities for ecucational training. 

AnotJ-jor advantage of the FAS is that instructions have been recorded 
on a tape. Thus, persons \nth poor reading sJJJJs should Jiave an advantage 
on this test^over traditio^ial paper-and-pencil tests. Reading ability does 
play a lar^^e part, hov/cv(:;r^ in being able to aasv/er questions on both the 
Verbal mid Numerical tests. 

What makes the FAS worth furthcy^ investigation despite tlie limitations 
discussed previously, is the use of items based on everyday experience. 
Such items should malic the test more interesting to adults who have very 
practical concerns and v. ant to see tlie relevancy of tests tliey are taking. 
In addition, the results pf the study reported Iiere suggest that the more 
relevmit everyday iter;s may be very importjint in tests of numerical ability, 
particularly v.lica testing diBadviuitaged persons. It should 1)0 noted, hov/ever, 
tlrcit although the. test was designed to contain Jiiaterial familiar to botli the 
advimtagcd ruid di;;iidvanlagcdj ru.sulLs of studies reported in the FAS numual 



reveal that differences in test performajice between Blacks and Wliites have 
not been eliminated. - r. 

Although there are limitations to the FAS, it is probably one of the 
best around for its staled purposes* Individual users should, however, con- 
duct their ov/n studies to see how useful the test is for their specific needs* 
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Performance of Educationally Disadvantaged Adults on Relevant 
Traditional Items for the Fundamental Achievement Series (Form B) 









Difficulty 


a 


Discrimination^ 


Items 


Options 


Description of Content 


Median 


Range 


Median 


Range 


Verbal test^ 


Relevant ^ 














1-6 


4 


Public signs 


80 


67 - 87 


.54 


. 46 - , 56 


7-8 


4 


Bus signs 


70 


62 - 77 


. 34 


.33 - .35 


9-13 


open 


Menu 


57 


13 - 74 


.55 


. 20 - . 56 


14-18 


12 


Apartment house list 


90 


79 - 93 


,50 


.41-67 


19-21 


9* 


Telephone book 


90 


80 - 92 


.54 


. 46 - . 54 


22-24 


open 


Copy sentences 


62 


61 - 70 


.57 


. 56 - . 57 


25-34 


open 


Announcements 


30 


16 - 48 


.38 


. 29 - . 42 


Traditional 












*- 


35-58 


2 


Spelling 


70 


32 - 95 


.50 


.02 - .69 


59-76 


4 


Picture vacabujary - 


91 


59 - 98 ^ 


.41 


.00 - .53 


7V-100 


4 


Synonymns 


59 


36 - 85 


.57 


.31 - .70 


Numerical test^ 


Relevant 














. 1 


4 


Number recognijtion 


93 


93 


. ^:6 


.46 


. 2-3 


4 ^ 


Time (clock) recognition 


89 


89 


.58 


. 57 - . 59 


4-9 


open 


General information 


61 


16 - 80 


.50 


. 44 - . 57 


10-13 


open 


Adding coin values 


89 


87 - 93 


.46 


. 43 - . 52 


14-22 


4 


General information 


75 


28 - 98 


.39 


-.12 - .54 


23 


4 


Reading a ruler 


33 


33 


*08 , 


.08 


24 


4 


Computing area 


11 


11 


.19 


.19 


25-26 


open 


Adding a restaurant bill 


55„ 


54 - 56 


-.12 


-.19 - (-).05 


27-28 


open 


V/riting a personal check 


36 


34 - 38 


.12 


.06 - .17 


29-31 


open 


Reading a calendar 


46 


36 - 56 


-.09 


-.10 - (-).08 


32-33 


open 


Reading a chart (graph) 


50 


43 - 56 


-.07 


-.09 - .05 


34-36 


open 


Reading a postal chart 


48 ' 


26 - 49 


.03 


.03 - .15 


37-38 


open 


Reading a mileage chart 


02 


00 - 04 


-.12 


'-.24 - .00 


39 


open 


Recognizing *'radius** 


10 


10 


.22 


.22 


Traditional 














40-59 


open 


Numerical computation 


33 


03 - ^6 


.05 


-.15 - .29 


60-69 

1, 


open 


Word problems 


12 


02 - 41 


.14 


.00 - .25 



^ Percent correct 
^ Polnt-biaerial r 
^Nr 61, x=: 66.62, 
^ — i-N-«-6Vx-"^40^59r 



SD=.19. 41 
SD-^t3..29^ 



13 



